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Moderation analysis urban school district were randomly assigned to receive training in the Incredible Years Teacher 


Classroom Management program (IY TCM), a classroom management intervention. We examined 
how teacher perceptions of their school environment moderated intervention effects for pre- 
viously established treatment outcomes — implementation of effective classroom methods, stu- 
dents' social behaviors, emotional regulation, and social competence. Results showed that 
treatment effects on teacher implementation and student outcomes were moderated by teachers' 
sense of affiliation to their school. Specifically, main effects on implementation of effective 
classroom management strategies were only observed among teachers whose perceptions of in- 
itial teacher affiliation was low or average; whereas main effects on student outcomes were only 
found for teachers with initial high levels of affiliation. 


1. Introduction 


Although there is a well-known evidence base for teacher use of effective classroom management practices, teacher im- 
plementation of these practices varies. Teachers report feeling under prepared in the area of classroom management and request 
professional development in this area (Reinke, Stormont, Herman, Puri, & Goel, 2011). As a result, several interventions focused on 
supporting teachers' classroom management skills have been developed. However, even when provided with training in effective 
classroom management strategies, there is variation in how teachers learn and adopt them (Reinke, Stormont, Herman, & Newcomer, 
2014). Lower fidelity can limit the impact of the intervention on student outcomes (Capin, Walker, Vaughn, & Wanzek, 2018; Carroll 
et al., 2007; Durlak & DuPre, 2008; O'Donnell, 2008). Many factors can influence the fidelity with which an intervention is im- 
plemented (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). A school's organizational health is one aspect of the broader social 
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environment that has been hypothesized to influence implementation of interventions (Bast, Due, Ersbgll, Damsgaard, & Andersen, 
2017; Domitrovich et al., 2008). The current study examined the association between organizational health indicators, teacher 
implementation of classroom practices, and student outcomes within the context of a large randomized control trial evaluating a 
teacher classroom management intervention. The following sections provide a review of the literature and study aims. 


1.1. The role of school psychologists in fostering organizational health 


School psychologists play a central role in fostering positive school environments that support effective instructional and man- 
agement practices. The NASP Practice Model specifies system-level supports as one of four broad areas of school psychology practice 
which includes knowledge of organizational contexts and theory (Skalski et al., 2015). In particular, school psychologists are skilled 
in advocating for and promoting policies and practices that create positive school climates and effective discipline strategies. 
Moreover, other domains of the Practice Model emphasize school psychologists roles in consulting and collaborating at the classroom, 
school, and systems level and gathering data to support fidelity of implementing effective practices and their outcomes. Given the 
profound influence that classroom environments have on student learning and development, one critical area of school psychology 
consultation includes knowledge about and skill in supporting effective teacher behavior management practices. Although much is 
known about effective classroom practices, the widespread variability in implementation of these practices suggests that school 
psychologists also need to be experts on organization factors that impede or promote these practices (Merrell, Ervin, & Peacock, 
2012). 


1.2. Classroom management practices 


Effective classroom management practices are critical for teaching and learning (Freeman, Simonsen, Briere, & MacSuga-Gage, 
2014; Korpershoek, Harms, de Boer, van Kuijk, & Doolaard, 2016) and have a number of obvious benefits for students, including 
reducing classroom disruption and problem behaviors (De Martini-Scully, Bray, & Kehle, 2000; Langland, Lewis-Palmer, & Sugai, 
1998; Musser, Bray, Kehle, & Jenson, 2001), promoting more on-task student behavior and engagement (Tobin & Capie, 1980), 
higher social and emotional competence (Reinke, Herman, & Dong, 2018), and better student achievement (Freiberg, Huzinec, & 
Templeton, 2009; Stronge, Ward, & Grant, 2011; Stronge, Ward, Tucker, & Hindman, 2007). A meta-analysis of classroom man- 
agement intervention studies showed consistent evidence for enhanced academic, behavioral, and socio-emotional outcomes 
(Korpershoek et al., 2016). Conversely, ineffective classroom behavior management practices can lead to more off-task behaviors and 
disruptive classrooms (Jones & Jones, 2004; Stronge et al., 2011), negative teacher-student interactions (Conroy, Sutherland, 
Haydon, Stormont, & Harmon, 2009; Reinke & Herman, 2002), and negative behavioral, social and academic outcomes (lalongo, 
Poduska, Werthamer, & Kellam, 2001; Webster-Stratton, Reid, & Hammond, 2004). These outcomes can be exacerbated in urban 
environments due to constraints such as larger classroom sizes, staffing issues, and fewer resources (Kwok, 2017). 

There is a strong research base on specific classroom management strategies that teachers can adopt in order to promote positive 
student outcomes such as decreases in student disruption, greater on-task behavior, better engagement, and improved academic 
performance (Conroy et al., 2009; Curby, Rimm-Kaufman, & Abry, 2013; Hutchings, Martin-Forbes, Daley, & Williams, 2013; Ialongo 
et al., 2001; Reinke et al., 2018). Simonsen, Fairbanks, Briesch, Myers, and Sugai (2008) identified 20 practices broadly that had 
sufficient evidence for adoption for classrooms. They classified these practices as those that (i) maximize structure and predictability, 
(ii) establish and provide feedback on expectations, (iii) actively engage students, (iv) use a continuum of strategies to acknowledge 
appropriate behavior, and (v) use a continuum of strategies to acknowledge inappropriate behavior. Specific examples of effective 
classroom management practices include providing high amounts of praise (Espin & Yell, 1994; Reinke, Lewis-Palmer, & Martin, 
2007) and pre-correction — which refers to providing students with reminders of appropriate behaviors or response prior to typical 
situations where problem behaviors occur (Colvin, Sugai, Good, & Lee, 1997; De Pry & Sugai, 2002; Smith, Lewis, & Stormont, 2011). 
Other strategies include finding ways to build relationships with students (Kwok, 2017), emphasizing student engagement with 
content (van Tartwijk, den Brok, Veldman, & Wubbels, 2009), and focusing on preventive rather than reactive management pro- 
cedures (Lewis & Sugai, 1999). 

Even though there is considerable research on specific classroom management strategies that can promote effective teaching and 
learning, teachers do not regularly use these strategies (Reinke et al., 2014; Reinke et al., 2018; Smith et al., 2011). Instead, teachers 
often emphasize control and discipline (Kaufman & Moss, 2010) and reactive strategies (Korpershoek et al., 2016; Shook, 2012). 
These practices may come from a lack of awareness about effective strategies (Korpershoek et al., 2016) as teachers do not receive 
adequate training in their preparation programs in classroom management skills (Begeny & Martens, 2006; Chesley & Jordan, 2012; 
Freeman et al., 2014; Hammerness, 2011; Kwok, 2017). Freeman et al. (2014) reported that there is a significant gap between the 
research base on effective management practices and teacher training experiences. Teachers feel unprepared to address the com- 
plexities of classroom management (Barrett & Davis, 1995; Houston & Williamson, 1993; Koehler, Feldhaus, Fernandez, & Hundley, 
2013), they rank classroom management as a major challenge, and cite it as a primary reason for leaving the profession early 
(Ingersoll & Smith, 2003; Kwok, 2017; Smith & Ingersoll, 2004). 

Outside of formal training there is a lack of support and training for handling student behavior and establishing effective 
classroom environments (Oliver & Reschly, 2007). Teachers report needing additional training in classroom management (Reinke 
et al., 2011; Reinke et al., 2014). There are a number of interventions or programs aimed at improving classroom management. These 
have a common set of core components such as using praise, contingent reinforcement, rules, precorrection, and social skills in- 
struction (Conroy et al., 2009). The Incredible Years Classroom Management Program is one such intervention that has substantial 
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support in research regarding its effectiveness. We describe this program briefly in the next section below. 
1.3. Incredible years classroom management program 


The Incredible Years Classroom Management Program (IY TCM) is a classroom management program for kindergarten through 
3rd grade teachers (see Webster-Stratton, Reinke, Herman, & Newcomer, 2011). During six full days of small group trainings led by 
certified IY TCM group leaders, teachers learn evidence-based classroom management practices (Simonsen et al., 2008). Specific 
strategies include proactive behavior management, which refers to practices that prevent disruptive behaviors before they can occur, 
praise and encouragement, problem solving, and approaches for increasing social competence and decreasing problem behavior 
(Webster-Stratton et al., 2004). The intervention is rooted in social learning theory (Bandura, 1977) and includes a heavy focus on 
video modeling to increase teacher learning of new skills. During each workshop, teachers view examples of real teacher student 
interactions and practice using these strategies. Between group training sessions, teachers meet with an IY TCM coach who conducts 
classroom observations, provides performance feedback, and supports teachers in implementing IY TCM strategies in their classrooms 
(Reinke, Stormont, Webster-Stratton, Newcomer, & Herman, 2012). 

The theory of change that underlies IY TCM proposes that training in the program leads to improved teacher classroom man- 
agement skills, particularly the use of proactive teaching practices, and more positive relations between teachers and students (see 
Webster-Stratton et al., 2004; Webster-Stratton & Herman, 2010). These proximal program effects on teacher behaviors in turn lead 
to reductions in student problem behaviors and increases in student prosocial and emotional regulation skills. In other words, changes 
in teacher behaviors, especially increasing positive and reducing aversive interactions with students, mediates the effects of IY TCM 
training on student social and behavior outcomes. 

Prior research on IY TCM indicates that exposure to IY TCM led to increases in teacher use of proactive teaching and more 
effective classroom environments (proposed mediators in the IY TCM logic model), as well as positive changes in student behavior 
(see Hutchings et al., 2007). Further, another study found that students in IY TCM classrooms were more on-task and teachers had 
fewer negative interactions with students (Hutchings et al., 2013). More recently, in a group randomized trial, Reinke et al. (2018) 
found that classrooms exposed to IY TCM experienced greater improvements in student prosocial behaviors, emotional regulation, 
and social competence. The intervention was especially beneficial for students with lower levels of social and academic competence 
at baseline. In the present study, we attempted to extend the findings from the Reinke et al. (2018) trial by examining the role of 
school organizational factors in moderating the previously reported main effects of IY TCM. 


1.4. Treatment fidelity and implementation barriers 


Adoption of evidence-based intervention programs depends on many factors including how much a school prioritizes classroom 
management in a list of diverse professional development needs (Maras, Splett, Reinke, Stormont, & Herman, 2014). Even when 
provided with training in effective classroom management strategies there is variation in how teachers learn and adopt these stra- 
tegies (Reinke et al., 2014). Fidelity of implementation refers to the extent to which the central components of an intervention are 
adopted by treatment participants as intended by providers (Abry, Rimm-Kaufman, Larsen, & Brewer, 2013). Overall, interventions 
that are adopted with higher fidelity have better outcomes (Capin et al., 2018; Carroll et al., 2007; Durlak & DuPre, 2008; O'Donnell, 
2008; Sanetti & Kratochwill, 2009). Further, examining fidelity of implementation can help describe the conditions under which a 
specific intervention can be effective (Abry et al., 2013; Stains & Vickrey, 2017) Many factors can influence the fidelity with which an 
intervention is implemented (Fixsen et al., 2005). These include school leadership support, teacher support, adequate resources, 
school personnel involvement in decision processes, quality of training, alignment with intervention with the rest of school orga- 
nization (goals, policies, programs), intervention visibility, personnel turnover, and school culture (Bast et al., 2017; Domitrovich 
et al., 2008; Forman, Olin, Hoagwood, Crowe, & Saka, 2008). Examples specific to classroom management interventions include the 
coaching received by teachers (Reinke et al., 2014) and follow-up support after intervention (Noell et al., 2005). 

Most efforts to overcome low fidelity of implementation have focused on individual teacher factors such as motivation, skill, or 
self-efficacy (Reinke, Lewis-Palmer, & Merrell, 2008; Sanetti, Collier-Meek, Long, Byron, & Kratochwill, 2015). Such approaches are 
limited as they consider one individual at a time and often ignore the socio-contextual factors that contribute to these individual 
factors. The organizational context of school is one such factor that can act as either a barrier or facilitator of effective intervention 
implementation (Anyon, Nicotera, & Veeh, 2016; Hall et al., 2014). We define school organizational factors as how a school as a 
social context organizes resources to support teaching and learning (Bryk, Sebring, Allensworth, Luppescu, & Easton, 2010; 
Danielson, 2002) This is distinct from a school's socio-economic and demographic background which includes characteristics such as 
school size, poverty among students served, and location of school (OECD, 2010, 2013). While there are studies on contextual 
influence of organizational factors specific to an intervention, such as administrator and teacher support and buy in for the inter- 
vention (Dijkstra, Walraven, Mooij, & Kirschner, 2017; Paxton, Wanless, & Rimm-Kaufman, 2013), the influence of intervention- 
independent aspects of context, such as the overall organizational health, is less well studied. 

Of the existing literature, several studies show that aspects of school organization are related to implementation fidelity (see for 
e.g., Beets et al., 2008; Gottfredson & Gottfredson, 2002; Gregory, Henry, & Schoeny, 2007; Henggeler, Schoenwald, Liao, 
Letourneau, & Edwards, 2002; Kam, Greenberg, & Walls, 2003; Payne, Gottfredson, & Gottfredson, 2006). On the other hand, 
Domitrovich et al. (2008) did not find a relationship between school level factors that included a measure of organizational health. As 
there is limited research on the influence of organizational conditions of implementation fidelity (Fixsen et al., 2005), more research 
is needed before valid conclusions can be made. 
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Furthermore, most studies examining the conditional influence of contextual conditions on fidelity to implementation are qua- 
litative studies or descriptive survey analyses (for a review see, McIntosh et al., 2014). Fixsen et al. (2005) notes that the contextual 
influence of leadership and other organizational factors are critical but understudied aspects of intervention implementation. 
Classroom interventions do not occur in isolation but within settings of organizations that are complex systems (Anyon et al., 2016; 
Fixsen et al., 2005); they are characterized by multiple levels of influence, diverse and often conflicting goals, loosely-coupled 
structures, competition for resources among political alliances, and unique organizational climates and cultures (Bolman & Deal, 
2013; Weick, 1976). Therefore, studying how interventions interact with these conditions to ultimately influence student outcomes is 
critical for intervention research. 


1.5. Organizational context 


Education researchers have developed numerous frameworks for conceptualizing how school organizational factors interact with 
one another and ultimately influence classroom instruction and student learning (see review by Hitt & Tucker, 2016). Two constructs 
that are commonly referenced across school organization frameworks are leadership and teacher community. Most school organi- 
zational frameworks begin with leadership which is conceived as the prime mover of school improvement (see Bryk et al., 2010; 
Knapp, Copland, & Talbert, 2003; Louis, Leithwood, Wahlstrom, & Anderson, 2010). More than four decades of research on school 
leadership has focused primarily on school principals, specifically on their instructional leadership role (Hallinger, 2005; Hitt & 
Tucker, 2016; Neumerski, 2013). Instructional leadership refers to the direct or indirect role of the school principal in supporting 
good instructional practices (Hallinger, 2005; Hallinger & Heck, 1998; Supovitz, Sirinides, & May, 2010). This body of research has 
consistently shown than leadership effects are largely indirect, mediated through multiple other organizational factors (Hallinger & 
Heck, 1996a, 1996b, 1998; Leithwood, Louis, Anderson, & Wahlstrom, 2004; Louis et al., 2010; Witziers, Bosker, & Kruger, 2003). 

Mediating factors linking leadership with student outcomes include the quality of professional development that the school offers 
its staff. Offering an intervention or training on effective classroom management practices can be considered as an example of a 
professional development program through which school leadership has an indirect influence on student learning. A few recent 
studies have examined the mediational relationship between school leadership, the quality of professional development programs, 
and student outcomes (see for e.g., Sebastian, Allensworth, & Huang, 2016; Sebastian, Allensworth, & Stevens, 2014; Sebastian, 
Huang, & Allensworth, 2017). Less well-studied are how leadership conditions could also moderate the effectiveness of an inter- 
vention. The success of professional development, such as a classroom management program, can depend on the degree of support 
offered by leadership. Fixsen et al. (2005) notes that “The importance of facilitative administration is often discussed, and rarely 
evaluated with respect to implementation outcomes” (p. 58). 

Teacher community is another critical organizational structure frequently referred to in school organizational literature. It de- 
scribes the degree to which teachers consider belonging to a professional community in the school, characterized by mutual colla- 
boration, shared responsibility for students, reflective practice, and shared norms about student outcomes (Grodksy & Gamoran, 
2003; Kruse, Louis, & Bryk, 1995; Louis, Marks, & Kruse, 1996). Referred to as Professional learning community (PLC), there is an 
extensive research base on the antecedents and effects of strong teacher communities in schools (Lomos, Hofman, & Bosker, 2011). It 
is often considered as another mediator linking leadership to achievement (see for e.g., Bryk et al., 2010), similar to professional 
development. PLC can also be regarded as a contextual variable that conditions the effectiveness of an intervention. Aspects of PLC 
that are specific to an intervention such as teacher buy-in (Anyon et al., 2016) have been studied before, but moderation studies that 
examine the importance of school-wide aspects of teacher community for intervention success, are absent. In the present study we 
will examine how these two organizational constructs—school leadership and teacher community, influence the extent to which 
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Fig. 1. Summary of strengths on five essential supports (Bryk et al., 2010) provided to each school in Chicago Public Schools. 
Source: https://consortium.uchicago.edu/(Sebring, Allensworth, Bryk, Easton, & Luppescu, 2006). 
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teachers adopt effective classroom management practices. 

A framework for urban school organization developed by Bryk et al. (2010), includes school leadership and teacher professional 
community among key essential supports driving instructional quality and student improvement. The other organizational supports 
are family involvement and supportive learning climate (Fig. 1). This framework was developed from a longitudinal study of 100 
public elementary schools in the large urban school district of Chicago to identify organizational factors that differentiated improving 
versus stagnant schools on student outcomes such as achievement and attendance. The framework has been important for for- 
mulating policy and school improvement efforts in Chicago; schools in the district receive individualized reports on their strengths in 
each organizational support (Fig. 1). Bryk et al. (2010) note the role of organizational supports is especially influential in highly 
disadvantaged urban schools where there is greater diversity in students' academic, psychosocial, and health needs. Further, unstable 
leadership (Stoelinga, 2008) and high teacher turnover (Allensworth, Ponisciak, & Mazzeo, 2009) are additional challenges faced by 
urban school districts like Chicago. Earlier, we noted that organizational constraints in urban schools can exacerbate the negative 
effects of ineffective classroom behavior management practices (Kwok, 2017). 

Given that implementation of interventions must be supported by strong organizational components over the long term (Fixsen 
et al., 2005), it seems likely that the objectives of classroom management training programs in urban schools will best be achieved 
when organizational supports such as leadership and teacher community are strong. The arguments for conditional influence of 
organizational supports, where intervention effects depend on levels of organizational support, lend themselves naturally to mod- 
eration analysis, which can examine interactions between levels of key organizational supports and intervention success. 

Examining the importance of organizational conditions warrants a brief discussion on measurement issues related to capturing 
group level constructs such as leadership and teacher community. Often, group/unit constructs are measured using individual 
perception data and aggregated to the unit level by various means (van Mierlo, Vermunt, & Rutte, 2008). The most popular mea- 
surement model used in school organization research is the referent-shift consensus model. Here, individual level responses are used 
to measure a group level construct; however, members are asked about a group level construct (e.g., overall teacher affiliation in a 
school), rather than an individual construct such as one's own affiliation to school (see (Chan, 1998; van Mierlo et al., 2008). Ina 
referent-shift consensus model, differences between individuals from the same group can be treated as measurement error; members 
may not be equally well informed about the construct being measured and therefore could provide imprecise information (van Mierlo 
et al., 2008). 

However, researchers have also argued that variability among group members could be of substantive interest rather than just 
being considered as measurement error (Cole, Bedeian, Hirschfeld, & Vogel, 2011; DeRue, Hollenbeck, Ilgen, & Feltz, 2010; Klein, 
Buhl Conn, Smith, & Sorra, 2001). Many school organizational constructs such as leadership and teacher community show greater 
variation within-schools than between-schools (see for e.g., Bryk et al., 2010). One way to examine within-school individual dif- 
ferences as meaningful variation is to use dispersion models where some measure of this variation itself is used as an outcome (Cole 
et al., 2011; van Mierlo et al., 2008). Variations in the forms of dispersion (e.g., bimodal, fragmented) may also be important to 
investigate as they have different effects on group functioning (DeRue et al., 2010). 

Experimental designs where randomization is conducted within-schools such as at the classroom level provide an interesting 
context to study the substantive importance of individual perceptions of group level constructs. First, treatment effects on individual 
perceptions of group constructs may more readily be interpreted to translate into school level effects for treatment. Second, ex- 
amining the interaction between treatment effects on student outcomes and teacher perceptions of group constructs (e.g., organi- 
zational health) can help further the discussion on the substantive importance of within-school variation of perception data on group 
constructs. For example, while there may be a collective sense of organizational health that influences teacher behavior, the more 
proximal influence on how the organizational health affects teacher actions is each teacher's own perception of the organization's 
qualities. Even in a building with an overall high level of organizational health, if a teacher in that building has a negative perception 
of the organization, those perceptions can override any broader sense of the organization held by others that could have implications 
for treatment effects on student outcomes. The present study utilizes data an experimental design to study the substantive importance 
of individual perceptions of group level constructs by examining how teacher perceptions of school organization can moderate 
treatment effects. 


1.6. The present study 


The purpose of the current study was to evaluate the conditional influence of perceptions of school organizational factors on 
teacher implementation of classroom management practices and student outcomes in an urban elementary school context. The 
current study was conducted in the context of an efficacy trial of an established effective intervention for promoting effective 
classroom management strategies, the IY TCM. Prior studies have shown that on average, teachers who received the training were 
more likely to have observable improvements in their classroom practices and that students in these classrooms had improved social 
and self-regulation capacities (Reinke et al., 2014; Reinke et al., 2018). Here we examine whether teacher perceptions of their school 
environment moderate teacher implementation practices. We hypothesized that teachers who perceived their schools as having 
higher levels of organizational health as evidenced by ratings of teacher community and leadership at baseline would have higher 
levels of implementation fidelity and better student outcomes. 

For measures of organizational conditions, we used subscales from the Organizational Health Inventory (OHI-E; Hoy & Feldman, 
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1987) to measure specific aspects of leadership and teacher community — collegial leadership and teacher affiliation. Previous studies 
have used OHI-E subscales as outcomes themselves, or as mediators of behavioral interventions (see for e.g., Bradshaw, Koth, Bevans, 
Ialongo, & Leaf, 2008), not as moderators of intervention effects. The choice of measures for describing the organizational context of 
a school, reflects our earlier literature review of research on school organization that highlights the role of school leadership and 
teacher professional community in promoting school improvement. Moreover, prior studies have found that school leadership and 
teacher affiliation are two aspects of organizational health consistently linked to important teacher and student outcomes (Bottiani, 
Bradshaw, & Mendelson, 2014; Pas, Bradshaw, & Hershfeldt, 2012). The purpose of this study is to describe possible contingency 
effects of school organizational factors on treatment effects, i.e., if treatment effects of IY TCM on instruction and student outcomes, 
depend on some levels of organizational support. Therefore, we chose outcome measures from this specific RCT of IY TCM that have 
already been discussed in the literature — proactive teaching strategies for classroom management (Reinke et al., 2014) and student 
outcomes of prosocial behavior, emotional regulation, and social competence (Reinke et al., 2018). 


2. Method and data 
2.1. Participants 


The current study examined data from 105 teachers and 1817 students from nine urban schools who participated in a group RCT 
of the IY TCM program (Reinke et al., 2018). Ninety-six percent of eligible teachers and 84% of eligible students enrolled in the 
project (four teachers and 350 students declined to participate). Teachers were randomly assigned to the intervention or to a wait-list, 
business as usual conditions. See Appendix A for information on student and teacher participation in the study. Teacher participants 
were predominantly female (97%) and White (75%; 22% African American and 3% other). On average, teachers had 11 years of 
teaching experience. The majority of student participants were males (52%) and African American students (76%; 22% White, and 
2% other). Additionally, 61% of students qualified for free or reduced lunch, and 9% qualified for special education. 


2.2. Intervention and data collection procedures 


After teacher and student recruitment, data were collected in each classroom early in the school year (October), before the 
intervention, and near the end of school year (April/May), post-intervention. The teachers assigned to the IY TCM intervention 
attended all-day group trainings in October, December and February. Two doctoral level IY TCM group leaders who had been trained 
and supervised by the program developer co-facilitated each training session. Nearly all intervention teachers attended all six 
workshops (attendance rate was 94-100% for each workshop). The few instances of missed workshops were addressed by having one 
of the trainers meet individually with the participant to go over missed material. Teachers were highly satisfied with the training 
(mean rating of 6.44 on a scale from 1 to 7 with high scores indicating greater satisfaction) and would recommend it to a colleague 
(6.75). 

In between the training workshops, one of the two lead trainers provided ongoing coaching to intervention teachers by observing 
their classrooms and meeting with them individually for up to 1 h ona weekly basis. Coaching is an integral component of the IY TCM 
intervention and is utilized to review workshop content, set goals for the use of classroom management strategies, provide perfor- 
mance feedback, model effective instruction, and help develop action plans (Reinke et al., 2014). All teachers received a minimum 
dose of coaching (i.e., four weekly visits), and all teachers had the option to request and receive more coaching than this minimal 
dose. The coach also differentiated coaching time beyond the minimal dose based on teacher need. During these meetings, a coach 
would review prior goals from the previous week and review performance feedback data collected during observations. This dis- 
cussion would review the teacher's use of proactive strategies and plans for refining existing strategies or the adoption of new ones. 
On average, a teacher spent about six hours with the coach in between the IY TCM trainings (Reinke et al., 2018). 

For more information about curriculum content see (http://incredibleyears.com/programs/teacher/classroom-mgt-curriculum). 
The IY TCM intervention was offered to the wait-list condition teachers at the end of the study, after follow-up data had been 
collected. During the study period, they continued business as usual teaching and professional development opportunities. 


2.3. Measures 


2.3.1. Organizational health 

Measures of organizational health were taken from the Organizational Health Inventory (OHI-E; Hoy & Feldman, 1987). The full 
OHI-E instrument is a 37-item instrument that measures teacher affiliation, academic emphasis, collegial leadership, resource in- 
fluence, and institutional integrity. Of these subscales we focused on two that tapped into school leadership and teacher community. 
Teacher Affiliation (a = 0.87) was measured by 9 items about a teacher's sense of connection with peers and the school and captures 
a sense of friendliness and strong affiliation with the school. Teachers with higher scores indicate a sense of liking one's peers and of 
shared accomplishment with them. These teachers find ways to accommodate to the routine and accomplish their jobs with en- 
thusiasm. Examples of items in this measure include: Teachers in this school like each other, Teachers exhibit friendliness to each 
other, and Teachers express pride in their school. Collegial Leadership (a = 0.92) includes 10 items about how friendly and sup- 
portive teachers perceive the school principal to be. Examples of items in this measure include: The principal explores all sides of 


139 


J. Sebastian et al. Journal of School Psychology 72 (2019) 134-149 


topics and admits that other opinions exist, The principal conducts meaningful evaluations, and The principal maintains definite 
standards of performance. The response categories for items for both measures were on an ordinal scale from 1: ‘rarely occurs’, to 4: 
‘very frequently occurs’. Previous studies support the construct validity of the OHI-E (see Hoy & Tarter, 1997). A prior study found that 
scores on the teacher affiliation and collegial leadership subscales were associated with both levels and changes in teacher burnout 
and self-efficacy (Pas et al., 2012). Items for the collegial leadership and teacher affiliation subscales were averaged to create a single 
score for each measure. 


2.3.2. Teacher implementation of proactive strategies 

The Brief Classroom Interaction Observation Revised observation code (BCIO-R; Reinke, Stormont, Herman, Wachsmuth, & 
Newcomer, 2015) was used to capture direct observations of teaching practices. The BCIO-R has high rates of interrater agreement 
and scores are associated with teacher self-efficacy and burnout (Reinke et al., 2015). Additionally, the BCIO-R subscales are sensitive 
to intervention effects (Reinke et al., 2014, 2015). Independent observers, blind to study condition, used the BCIO-R to gather the 
frequency of proactive classroom management strategies and reactive strategies. All observations were conducted while the teacher 
was delivering math or reading instruction. Observers were graduate and undergraduate students in education or social science fields 
who had attended a full day training on the BCIO-R (including coding video examples) and then practiced coding in real classrooms 
until they attained high levels of agreement (85% or higher) with master coders. These observations were conducted at 4 time points, 
first in October before teachers received IY TCM training, the second and third were in between training and coaching sessions, while 
the final observation was at the end of school year, and followed completion of all training/workshop sessions and all coaching 
(Reinke et al., 2014). For the purposes of this study, we used observations collected at baseline (pre-training) and at the end of the 
year (post training). The pre and post observations were aggregated from a series of 5-min observation recordings. The lengths of 
these aggregated recordings from pre-observation recordings (M = 41.26) and post-observations (M = 36.27) were comparable. Each 
observation period included information about the rate of overall praise, precorrections, and reprimands. The ratio of the rate of 
positive strategies (praise and precorrections) to the rate of total observed strategies (praise, precorrections, and reprimands) was 
used as an indicator of teacher implementation practices. Reliability checks for the observations were conducted for a sample of the 
observations (30%), which found that the percentage of agreement between raters on the BCIO-R, calculated with second-to-second 
comparisons of raters, was 91%; the acceptable reliability for this instrument is 80% (Reinke et al., 2014). 


2.3.3. Student outcomes 

In this study, we examined the moderating influence of school organizational factors on intervention effects on three outcomes — 
emotional regulation, prosocial behavior, and social competence, to examine if treatment effects depended on initial perception levels 
of organizational health. Similar to the observation outcome, we compared these measures between the pre-training and post-training 
time points. 

The Teacher Observation of Classroom Adaptation-Checklist (TOCA-C; Koth, Bradshaw, & Leaf, 2009) is a 54-item measure of 
student behavior. Teachers were asked to rate each student on the items referencing the past three weeks. The subscales of the TOCA- 
C included in the present study are Emotional Dysregulation and Prosocial Behavior. The item responses ranged from 1 (never) to 6 
(almost always). Prior studies support the construct validity and factor structure of the TOCA-C (Koth et al., 2009; Kourkounasiou & 
Skordilis, 2014), as well as strong evidence of subscale predictive validity; for instance, both scales predict the frequency of office 
discipline referrals (Pas, Bradshaw, & Mitchell, 2011) and both are sensitive to intervention effects (Bradshaw, Waasdorp, & Leaf, 
2012). The reliability of Emotional Dysregulation subscale was 0.80 in the pre-training survey and 0.85 in post-training survey. The 
reliability of the Prosocial behavior subscale was 0.90 in the pre-training survey and 0.92 in post-training survey. 

The Revised Social Competence Scale-Teacher version (T-COMP; Gifford-Smith, 2000) assesses the teacher's perception of a 
student's overall social competence. Teachers were asked to rate 17 items for each student in comparison to other children in the 
same grade. Item responses range from 0 (almost never) to 5 (almost always). The T-COMP scales have a strong internal consistency 
and a consistent factor structure; additionally, scores distinguish between high risk and normative samples (Gifford-Smith, 2000). 
Additionally, teacher ratings were found to be significantly related to mother-reported social competence (Webster-Stratton & 
Lindsay, 1999). The reliability of social competence subscale on T-COMP was 0.95 in the pre-training survey and 0.96 in post-training 
survey. 

Prior efficacy studies of IY TCM found that the intervention improved student emotional regulation (d = — 0.14), prosocial 
behavior (d = 0.13), and social competence (d = 0.13); further, the intervention had stronger effects on students initially low on 
social competence (Reinke et al., 2018). 


2.4, Analysis plan 


The analysis plan for this study involved two steps: (i) aggregating student level measures to the classroom/teacher level using 
hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002), and (ii) moderation analysis using PROCESS (Hayes, 2013). Among 
the outcomes we considered for this study, some of them were measured at the student-level - prosocial behavior, emotional reg- 
ulation, and social competence. While these measures were student specific, the randomization and intervention were conducted at 
the classroom-level. Before conducting moderation analysis, the student measures were aggregated to the classroom-level using 
Hierarchical Linear Modeling (HLM) and then the classroom-level residuals were saved. The following model was used to obtain 
classroom-level estimates of student outcomes. 


140 


J. Sebastian et al. Journal of School Psychology 72 (2019) 134-149 


M WwW 
(Collegial Leadership) (Teacher Affiliation) 


x Y 
(Intervention Status) (Outcomes) 


Fig. 2. Conceptual diagram of the moderation model. 


Level 1 (Student): 


Post Observation Score, = Boj + By * (Pre Observation Score) + hj (1) 


Level 2 (Classroom): 


Boj = Yoo + Hai; (2) 


By = No (3) 


The above model is a random intercept model, i.e., only the intercept which represents the expected post-treatment score on a measure 
conditional on its pre-treatment score, is allowed to vary between classrooms. This is an alternative to simply averaging the measures to the 
classroom-level. With this procedure, we obtain averages that are weighted by reliability while also partialing out the variation from pre- 
treatment scores of that measure (Raudenbush & Bryk, 2002). Classroom level residuals (ug) were saved from these analyses for prosocial 
behavior, emotional regulation, and social competence. Teacher implementation of proactive strategies was a teacher/classroom level 
measure and did not require aggregation. We regressed post-treatment scores of this measure on the corresponding pre-treatment scores and 
saved the residuals; these were then used in the moderation analysis. We also conducted simpler analyses where: (i) the student level 
outcomes were averaged to the classroom level using simple means, and (ii) outcomes were not adjusted for pre-treatment levels, the pre- 
treatment measure was simply entered as a covariate during the moderation analysis. The results were similar across these different sets of 
analyses; we reported here the results from averaging measures using HLM. The final moderation analysis using PROCESS included all 
teachers who had valid outcome measures as well as data on their pre-treatment measures; there was very little missing data (see Table 1). 

The main aim of this study was to examine how treatment effects on the implementation of effective classroom management strategies 
and student social and emotional outcomes were conditional on perceptions of initial organizational conditions. For this purpose, we con- 
ducted moderation analysis using the SPSS macro PROCESS version 2.16 (Hayes, 2013). As described earlier, we first aggregated student 
level measures to the classroom/teacher level, as the current version of PROCESS cannot conduct multilevel modeling. For each outcome, we 
included two moderators — collegial leadership and teacher affiliation. PROCESS provides various built in model templates, from which 
researchers can select the appropriate model that is most useful to answer their research questions. The template used in this study to conduct 
moderation analysis is shown in Fig. 2, which is a simple moderation model and allows for the inclusion of two moderators. This model 
examines if the effect of IY TCM treatment (X) on different outcomes (Y) - teacher implementation of effective management strategies, and 
student outcomes, depended on various levels of two moderators — teacher affiliation (M) and collegial leadership (W). We controlled for 
grade level and school effects in the moderation analysis using the fixed effects approach, by including dummy variables for each grade and 
school. The analysis can therefore be considered as examining within-school variation for each outcome. 


3. Results 


Table 1 provides the descriptive statistics for the study variables. Recall that the student measures were aggregated to the 
classroom level using HLM; residuals saved from HLM were used for the moderation analysis. These residuals are centered at the 
means of each outcome. Table 2 describes the effects of the treatment on teacher implementation of effective classroom management 
conditional on various levels of the two moderators - individual perceptions of teacher affiliation and collegial leadership. The 
outcome here is the percent of observed proactive strategies used by the classroom teacher. Prior to the moderation analysis, the post- 
treatment measure was regressed on the pre-observation scores. Therefore, the estimates shown here reflect the percent difference in 
proactive strategies employed, between treatment and control groups, holding constant pre-treatment implementation scores. 
PROCESS displays conditional effects for three levels of each moderator, at the mean, at one standard deviation below the mean, and 
at one standard deviation above the mean. We centered the moderators in PROCESS. 


141 


J. Sebastian et al. Journal of School Psychology 72 (2019) 134-149 


Table 1 

Descriptive statistics of variables. 
Variable N Mean SD Min Max 
Collegial leadership (OHI-E) 103 2.80 0.69 1.20 4.00 
Teacher affiliation (OHI-E) 104 2.91 0.53 1.00 4.00 
Prosocial behavior (TOCA-C) 104 0.00 0.36 —0.92 0.89 
Emotional regulation (TOCA-C) 104 0.00 0.19 —0.56 0.54 
Social competence (T-COMP) 104 0.00 0.31 —0.73 0.94 
Proactive strategies (BCIO-R) 104 0.00 19.26 — 45.92 35.48 


Note. All measures are residuals from regression models. These residuals are centered around zero. 


Since there are two moderators, the results are displayed for nine different levels of these two moderators. For example, the first 
three rows of Table 2 show treatments effects holding teacher affiliation constant at one standard deviation below its mean, while the 
measure of collegial leadership is at — 0.69, 0.00, and 0.69, which are values of collegial leadership one standard deviation below the 
mean, at the mean, and one standard deviation above the mean, respectively. The first row of Table 2 shows that in schools that were 
1 standard deviation below the mean on both teacher affiliation and collegial leadership, the percentage of proactive strategies 
employed by IY TCM teachers was 18.58% more than control group teachers. The last row of Table 2 shows that for teachers who are 
one standard deviation higher than the mean on both teacher affiliation and collegial leadership, there was no significant difference 
between treatment and control group teachers. The results show that the IY TCM treatment showed positive effects on teacher 
implementation of proactive strategies for teachers who reported low or average levels of teacher affiliation. Further, higher 
treatment effects were observed at lower levels of initial teacher affiliation, holding collegial leadership constant. 


Table 2 
Conditional treatment effects of teacher implementation of proactive strategies. 
Outcome Teacher Affiliation Collegial Leadership Estimate SE Lower Level CI Upper Level CI 
Percent of observed proactive strategies -0.54 -0.69 18.58** 6.02 6.61 30.55 
-0.54 0.00 17.14** 5.60 6.00 28.28 
-0.54 0.69 15.70 7.95 -0.12 31.51 
0.00 -0.69 14.52* 5.59 3.41 25.63 
0.00 0.00 13.08** 3.77 5.59 20.57 
0.00 0.69 11.64* 5.82 0.06 23.22 
0.54 -0.69 10.46 7.83 -5.10 26.03 
0.54 0.00 9.02 5.66 -2.24 20.28 
0.54 0.69 7.58 6.30 -4.94 20.10 


Note. * p < .05; ** p < .01; *** p <.001. 


Table 3 describes conditional treatment effects on student outcomes — prosocial behavior, emotional regulation, and social competence. 
For these outcomes, treatment effects were observed only among teachers’ whose perceptions of initial teacher affiliation levels were average 
or high; perceptions of initial collegial leadership levels appeared less influential as treatment effects are observed at all levels of initial 
collegial leadership. For instance, the first nine rows of Table 3 describe treatment effects for various levels of teacher affiliation and collegial 
leadership for the outcome of prosocial behavior. The first three rows show that there are no treatment effects when holding teacher 
affiliation constant at one standard deviation below the mean (— 0.54), while varying the values of collegial leadership between — 0.69 (one 
standard deviation below the mean), 0.00 (at the mean), and 0.69 (one standard deviation above the mean). Treatment effects are observed 
when teacher affiliation is at the mean 0.00 (rows 4-6), and one standard deviation above the mean, 0.54 (rows 7-9). Similar patterns are 
observed for the outcomes of emotional regulation, and social competence. 


Table 3 
Conditional treatment effects on ratings of student outcomes. 
Moderators 
Outcome Teacher Affiliation Collegial Leadership Estimate SE Lower Level CI Upper Level CI 
Prosocial Behaviors (TOCA: Prosocial Behaviors) -0.54 -0.69 0.07 0.09 -0.11 0.25 
-0.54 0.00 0.03 0.08 -0.14 0.19 
-0.54 0.69 -0.02 0.12 -0.26 0.22 
0.00 -0.69 0.18* 0.08 0.01 0.35 
0.00 0.00 0.13* 0.06 0.02 0.24 
0.00 0.69 0.08 0.09 -0.09 0.26 
0.54 -0.69 0.28* 0.12 0.05 0.52 
0.54 0.00 0.24* 0.09 0.07 0.40 
0.54 0.69 0.19* 0.09 0.00 0.38 


(continued on next page) 
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Table 3 (continued) 


Moderators 

Outcome Teacher Affiliation Collegial Leadership Estimate SE Lower Level CI Upper Level CI 

Emotional Regulation (TOCA: Emotional Dysregulation)  -0.54 -0.69 -0.04 0.06 -0.15 0.07 
-0.54 0.00 -0.03 0.05 = -0.13 0.07 
-0.54 0.69 -0.03 0.07 -0.17 0.12 
0.00 -0.69 -0.11* 0.05 = -0.21 0.00 
0.00 0.00 -0.10** 0.03 -0.17 -0.03 
0.00 0.69 -0.10 0.05 -0.20 0.01 
0.54 -0.69 -0.17* 0.07 -0.32 -0.03 
0.54 0.00 -0.17** 0.05 = -0.27 -0.07 
0.54 0.69 -0.17** 0.06 -0.28 -0.05 

Student Competence (T-COMP: Social Competence) -0.54 -0.69 0.07 0.09 -0.11 0.24 
-0.54 0.00 0.04 0.08 -0.13 0.20 
-0.54 0.69 0.01 0.12 -0.22 0.24 
0.00 -0.69 0.18* 0.08 0.02 0.35 
0.00 0.00 0.16** 0.06 0.05 0.27 
0.00 0.69 0.13 0.09 -0.04 0.30 
0.54 -0.69 0.30* 0.12 0.07 0.53 
0.54 0.00 0.27** 0.08 0.11 0.44 
0.54 0.69 0:25** 0.09 0.06 0.43 


Note. * p < .05; ** p < .01; *** p <.001. 


4. Discussion 


The present study examined whether treatment effects of a classroom management intervention, IY TCM, on teacher instruction 
and student outcomes were contingent on school organizational conditions, focusing on two variables — teacher affiliation and 
collegial leadership. The data used for this study came from a randomized efficacy trial of IY TCM. We chose to focus on outcomes 
from this specific trial that have already been published and used the SPSS macro, PROCESS to conduct the moderation analysis 
(Hayes, 2013). Most frameworks of school organization emphasize the contingent or conditional nature of school organization in how 
they influence school processes including the adoption and implementation of effective school practices (Bryk et al., 2010; Hitt & 
Tucker, 2016). Yet, research using moderation analysis to examine the conditional influence of organizational factors on school 
processes such as proximal and distal intervention effects, is limited. This study contributes to the limited empirical research base that 
examines conditional effects of school organization on the uptake of proactive teaching practices and changes in student outcomes in 
the context of a professional development and coaching intervention. 

Of the two moderators we examined, teacher affiliation appeared to be the more influential organizational condition for influ- 
encing treatment effects. This could be because teacher affiliation is more closely related to teacher practice and classroom in- 
struction than leadership conditions. Most theoretical frameworks of school organization also support mediated effects of school 
leadership on instruction and student outcomes (Hitt & Tucker, 2016). Further, empirical research on the topic of school leadership 
research has consistently shown that leadership effects are largely indirect, mediated through other school organizational factors 
including teacher community and school climate (Hallinger, 2005; Hallinger & Heck, 1996b, 1998; Leithwood et al., 2004; Louis 
et al., 2010). This could explain why initial levels of collegial leadership did not moderate treatment effects similar to teacher 
affiliation. More complex mediated-moderation models might be useful to accurately capture the influence of school leadership. 

Consistent with our hypotheses, we found that organizational health moderated treatment effects. Specifically, improvements in 
student outcomes were only found for teachers with average to high positive perceptions of organization health at the beginning of 
the year. This finding adds some nuance to previously reported main effects of this specific intervention, namely only teachers with 
average/high levels of affiliation at the outset of training were likely to achieve student level impact by the end of the year. Contrary 
to our hypotheses, only teachers with average or below average levels of perceived teacher affiliation at baseline increased their use 
of proactive teaching strategies. In other words, no changes in teacher implementation of proactive strategies were observed among 
teachers who reported high levels of initial teacher affiliation. Overall, perceptions of teacher affiliation were more important in 
influencing classroom and student level treatment effects of IY TCM in comparison to perceptions of collegial leadership. 

Although the finding that low teacher affiliation was associated with higher use of proactive classroom strategies was unexpected, 
it may be explained in the context of a widely used framework for understanding teacher implementation practices (Han & Weiss, 
2005). In this framework, teacher affiliation would be conceptualized as a pre-implementation phase factor that would make it more 
or less likely that teachers would implement a new practice. In the present study, we found that a pre-implementation phase barrier to 
implementation (low affiliation) was actually associated with greater improvements in classroom atmosphere. One explanation for 
this may be the nature of the intervention. IY TCM is group delivered and is designed to foster positive relations among teacher 
participants by way of buddy systems, group support, and encouragement. Thus, the structure and format of IY TCM may help 
overcome a hypothesized pre-implementation barrier (low affiliation) by building in implementation processes for fostering greater 
affiliation (at least to the teachers in the training group many of whom were from the same school) and skills that fostered more 
motivation in the feedback motivation loop conceptualized by Han and Weiss (2005). A second explanation that we considered was 
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that low affiliation teachers had more room for growth or improvement in their classroom management practices. However, we ruled 
out this explanation by finding that low and high affiliation teachers had comparable levels of proactive teaching skills at baseline. 

Although low teacher affiliation at baseline was associated with improved treatment effects on effective classroom management 
practices, it was not a pre-condition for treatment effects on student outcomes. Instead, as predicted, main effects on student out- 
comes were conditional on high teacher affiliation at baseline. It was surprising that teachers with high affiliation did not also have 
improved implementation practices as initially we expected these classroom management variables to be a pre-condition for treat- 
ment effects. One possibility is that teachers with high affiliation were already employing higher rates of proactive strategies before 
treatment. However, in our analysis we controlled for pre-treatment levels of proactive classroom management strategies. Therefore, 
the observed treatment effects are for teachers who were similar at pre-treatment in their use of proactive strategies. Moreover, the 
correlation between pre-treatment levels of teacher affiliation and use of proactive management practices was low (r = 0.12, 
Pp = .21). Deprivatized practice, or the sharing of best practices among teachers, is one of the hypothesized benefits of developing 
teacher communities in schools (Bryk, Camburn, & Louis, 1999). Louis and Marks (1998) for example, found that teacher professional 
community was related to classroom organization characteristics. Their measure of teacher community included items that directly 
asked about deprivatized practice (peer observation), shared focus on student learning, and reflective dialog and planning around 
student learning progress. The OHI-E measure of teacher affiliation that we used to represent teacher community is more focused on 
teacher relationships to each other and the school. This difference in the instructional focus of the teacher community measures could 
explain why we did not find a relationship between teacher affiliation and use of proactive classroom management practices. 

A key assumption underlying the logic model for IY TCM was that teacher implementation practices would mediate treatment 
effects on student outcomes. Given the unexpected findings that low affiliation was associated with higher implementation practices 
but not student outcomes, and treatment effects for high affiliation teachers were only observed for student outcomes, we conducted 
post-hoc mediation models to examine whether our initial assumption was correct (see Appendix B). Findings indicated that teacher 
positive implementation practices did not mediate treatment effects on student outcomes. That is, although the intervention created 
teacher- and student-level improvements, the effects examined here were independent of each other. This unexpected finding may be 
attributed to the lack of a treatment main effect on student disruptive behavior (Reinke et al., 2018). In general, research and theory 
suggest proactive teaching and positive climate, the teacher implementation factors examined in this study, are most strongly as- 
sociated with a reduction in disruptive behaviors. Here the primary treatment effects were on student skill development in the 
domains of social skills and emotional regulation. These skill areas are likely rooted in other teacher behaviors besides the ability to 
alter classroom climate. 

Thus, it appears that the intervention impacted some other aspect of teacher behavior besides proactive classroom management 
that influenced student outcomes for those with high levels of teacher affiliation. It is important to note in this regard that IY TCM is a 
complex and multifaceted intervention. In addition to targeting proactive classroom management, the IY TCM also trains teachers to 
deliver social emotional coaching to students, work effectively with parents of students, cope with their own stress, and devise 
strategic behavior support plans for students with behavior problems. Each of these teacher skill development areas may be a 
reasonable candidate to explain how teachers with high initial affiliation impacted student learning of prosocial and emotional 
regulation skills. For instance, teachers with high affiliation may have been more likely and able to learn and implement the ad- 
vanced social emotional coaching skills taught in IY TCM and these skills would have enabled them to teach and support their 
students in developing emotional regulation and prosocial skills. Similarly, high affiliation teachers may have been more likely to 
engage with parents and foster social learning across home and school contexts. Skillful development of effective behavior support 
plans which include strategies for teaching and encouraging prosocial skills may also explain how high affiliation teachers impacted 
students. A future study will be needed to determine if teachers with high affiliation were more likely to engage in one or more of 
these strategies, which in turn increased the likelihood of favorable student outcomes. 

Contrary to our expectations, we found that teacher perceptions of principal leadership did not moderate intervention effects on 
teacher practices or student outcomes. Both existing theory (e.g., Bryk et al., 2010) and evidence (Bottiani et al., 2014; Pas et al., 
2012) suggest that staff perceptions of principal leadership behaviors are linked to school practices. Here we found that teacher 
implementation of classroom management skills and student prosocial and emotional regulation development were non-contingent 
on principal leadership perceptions. On the one hand, this finding can be interpreted positively as it implies that implementing a 
program like IY TCM will be unaffected by the quality of building leadership and thus teachers will be equally likely to have high 
quality implementation in schools with poor leadership settings. On the other hand, the finding implies that improving principle 
leadership is not a lever for better implementation. It is important to note, the study relied on a limited range of leadership behaviors. 
Although the scale used in the present study had been found to predict meaningful teacher and student outcomes, a longer scale 
tapping a broader array of leader behaviors may yield different effects. 

In the present study, our design focused on within school random assignment to condition and within-school variation of teacher 
perceptions of leadership and affiliation. While it is not known how these findings would extrapolate to whole-school implementation 
and variations in these constructs, it is reasonable to expect comparable intervention effects for main student outcomes. It is possible 
even stronger effects might be observed in the context of a school level randomization given the institutional and relational supports 
that would exist in such implementation. For instance, in a within school randomization design such as the current study, coaches 
worked with individual teachers and solved individual classroom and motivational barriers with each teacher. In a whole school 
implementation context, the entire building would be working on the same approach to classroom management. Teacher support 
might also increase since they could share resources and ideas with all teachers (compared to this study in which intervention 
teachers were told not to share materials or treatment principles with control teachers), and principal leadership could be brought to 
bear in moving classrooms in ways that were not possible in the present study design (e.g., principals were not trained in the method 
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whereas in a school level randomization as well as typical school practices, principals would knowledgeable about classrooms and 
align the feedback and support they give teachers with the school-wide practice model). 

We can only speculate how the moderation effects observed here would extrapolate to whole school contexts. Here we focused on 
individual level perceptions of leadership and affiliation as key drivers of implementation practices. Clearly, individual perceptions 
are related to building-level aggregations of these perceptions, and it is these aggregate values that are often used to define orga- 
nizational health. Thus, we would predict that school level affiliation would moderate teacher implementation of classroom man- 
agement practices. Similar to our explanation for within-school findings, the interventions focus on building teacher support and 
relations might help overcome low affiliation as a pre-implementation barrier and lead to school wide improvements in proactive 
teacher practices. In turn, we would expect teachers in schools with higher levels of affiliation to be more likely to achieve student 
level outcomes by the end of the year than teachers in schools with low affiliation. Further research is needed to examine these 
expectations developed from examining within-school relationships. 


4.1. Implications 


The findings from the current study may be useful in supporting school psychologists' roles as system-change agents. Two possible 
practical implications of the findings align with the primary influences on teacher perceptions of affiliation, the school context and 
individual interpretative bias. First, given that building aggregate scores of affiliation have been found to predict education outcomes 
(Bottiani et al., 2014; Pas et al., 2012; Sebastian et al., 2017), strategies to foster building-level affiliation could yield greater 
likelihood of impacting student outcomes over time. Second, teacher perceptions of affiliation are malleable at the individual level as 
well. Our findings call attention to the fact that individual perceptions of teacher affiliations can be influenced both by the actual 
context of schools and by individual perceptions, which in turn can influence the context (i.e., negative attitudes and beliefs beget 
negative attitudes and beliefs). Cognitive behavioral interventions are well-established approaches for making individuals more 
aware of their interpretative biases and restructuring them to be more adaptive (Herman & Reinke, 2015). That is, when building- 
level changes in affiliation are difficult to implement, individual interventions to alter perceptions of affiliation may be effective as 
well. School psychologists working with individual teachers could use some of these cognitive coping methods to assist teachers in 
arriving at more adaptive views of their school context (e.g., by paying attention to positive aspects of the environment, increasing 
their positive interactions with peers, and reinterpreting peer or leadership actions; see Herman & Reinke, 2015). The results of our 
study suggest that integrating these methods with instructional interventions such as classroom management programs, could im- 
prove intervention effectiveness. Such approaches may be particularly useful in urban setting which face contextual constraints that 
directly relate to teacher affiliation such as high teacher turnover. Shernoff et al. (2011) note that teachers' sense of affiliation and 
their effectiveness in classroom management are the strongest predictors of teacher attrition in urban settings. It follows then that 
classroom management interventions that include components linking to coping and increasing affiliation will have higher likelihood 
of success. 


4.2. Limitations 


The present study was conducted using data from an efficacy trial of IY TCM. However, the moderating conditions were not 
manipulated; thus causal inferences regarding moderating conditions are inappropriate. The data for this study came from an urban 
setting and therefore the results are not generalizable to other contexts. Additionally, the study relied on teacher reports of their 
perceptions of the organizational conditions. Although this is a common approach for measuring organizational health, additional 
measurement such as direct observations of these conditions would help validate whether these ratings are rooted in objective aspects 
of the environment, teacher bias, or some combination of both. It is possible that the unexpected findings may be due in part to 
differences in teacher perceptions of student behaviors over time and actual moment-to-moment behaviors in the classroom. 
Regardless, it is important to note that the teacher rating scale used in the present study, the TOCA, is a well-established tool for 
predicting short- and long-term youth behavioral outcomes. For instance, TOCA scores are significantly correlated with office dis- 
cipline referrals (Pas et al., 2011) and ratings in elementary school have been shown to predict mental health symptoms and di- 
agnoses, service use, school dropout, and violent behavior and arrest later in development (Bradshaw, Buckley, & Ialongo, 2008; 
Petras, Chilcoat, Leaf, Ialongo, & Kellam, 2004; Reinke, Herman, Petras, & Ialongo, 2008; Schaeffer, Petras, Ialongo, Poduska, & 
Kellam, 2003). Thus, intervention and moderation effects on teacher perceptions of youth development are noteworthy and 
meaningful. 

In this study, we also focused on individual teacher perceptions rather than aggregated building-level perceptions as is how much 
prior research has been conducted. Although this is a difference with prior research, we view it as a strength and avenue for future 
research given the potential implications for intervening at the perception level. The current study looked at the frequency of 
proactive behavior management strategies which does not fully capture the quality in adoption of classroom management practices. 
Future work can consider other methods of recording the quality of teaching strategies that could also help to explain the findings 
from moderation analyses we conducted for this study. Last, we chose to focus on outcomes from a specific trial of IY TCM that were 
already established in literature; future work can examine moderation effects on other outcomes as proposed by the intervention's 
logic model. While this work examined moderation of direct treatment effects, PROCESS can be used to study how organizational 
conditions moderate mediational relationships (moderated-mediation), which can also be a focus of future research. 
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5. Conclusion 


Teacher perceptions of peer affiliation but not principal leadership were linked to teacher implementation practices. Given the 
high interest and focus on explaining variations in real world implementation, teacher affiliation is a promising candidate for un- 
derstanding and supporting positive changes in teacher classroom behavior and student outcomes. Further, understanding pre- 
implementation factors, such as the organizational health indicators in this study can support future intervention efforts toward 
bridging the gap between research and practice by removing barriers to implementation with high fidelity in our schools. 


Appendix A. IY TCM participation flowchart 


Assessed for eligibility 
Teachers (n=109) 
Students (n=2168) 


Excluded (declined to participate) 
Teachers (n=4) 
Students (n=350) 


Classrooms Randomized 
Teachers (n=105; 96%) 
Students (n=1817: 84%) 


Allocated to lY TCM Allocated to Wait-List Control 
Teachers (n=53) Teachers (n=52) 
Students (n=900) Students (n=917) 


Lost to follow-up 
Teachers (n=0) 
Students (n=70; moved) 


| 


Analyzed Analyzed 
Teachers (n=53) Teachers (n=51) 
Students (n=833) Students (n=847) 


Lost to follow-up 
Teachers (n=0) 
Students (n=67; moved) 


Source: Reinke et al., 2018. 


Appendix B. Treatment effects mediated by classroom atmosphere and proactive strategies 


Outcome Estimate SE Lower level CI Upper level CI 
Prosocial behaviors (TOCA: prosocial behaviors) Mediator: classroom atmosphere —0.01 0.02 — 0.08 0.01 
Mediator: proactive strategies —0.01 0.03 —0.06 0.04 
Emotional regulation (TOCA: emotional dysregulation) Mediator: classroom atmosphere 0.00 0.01 —0.01 0.04 
Mediator: proactive strategies 0.00 0.01 —0.03 0.03 
Student competence (T-COMP: social competence) Mediator: classroom atmosphere 0.00 0.02 —0.03 0.05 
Mediator: proactive strategies 0.00 0.02 —0.05 0.05 
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